Neural Nets and RBF Nets

Declaration All sentences or passages quoted in this dissertation from other people's work have been specifically acknowledged by clear cross-referencing to author, work and page(s). Any illustrations which are not the work of the author of this dissertation have been used with the explicit permission of the originator and are specifically acknowledged. I understand that failure to do this amounts to plagiarism and will be considered grounds for failure in this dissertation and the degree examination as a whole. 1.1 Abstract This paper is experimental in nature and focuses on the strengths and weaknesses of a recently proposed boosting algorithm called AdaBoost. Starting with a literary review of boosting, we provide an in depth history of the algorithms that lead to the discovery of AdaBoost and comment on recent experimentation in this area. Boosting is a general method for improving the accuracy of a given learning algorithm and when used with neural nets, AdaBoost creates a set of nets that are each trained on a different sample from the training set. The combination of this set of nets may then offer better performance than any single net trained on all of the training data. The later half of the paper looks at what factors affect the performance of the algorithm when used with neural networks and radial basis function networks and tries to answer the following questions: (1) Is AdaBoost able to produce good classifiers when using ANN's or RBF's as base learners? (2) Does altering the number of training epochs affect the efficiency of the classifier when using ANN's or RBF's as base learners? (3) Does altering the number of hidden units have any effect? (4) How is AdaBoost affected by the presence of noise when using ANN's or RBF's as base learners and (5) What causes the observed effects? Our finding support the theory that AdaBoost is a good classifier for low noise cases but suffers from overfitting in the presence of noise. Specifically, AdaBoost can be viewed as a constraint gradient descent in an error function with respect to the margin, 1.3 Time Plan 1.3.1 Semester 1 Week1-3: Read papers on or around topic to gain a feel for what is involved and write interim report1. Week4-6: Learn how to use Matlab and experiment with AdaBoost implementation by Gunnar Raescht. Week7: Ensure that data sets to be used in empirical study are in the correct format. Week8/10: …